Information exchange display using color space encoded image

ABSTRACT

Embodiments of the present invention include systems, methods, and non-transitory computer program products for information exchange using color space encoded images. A color space encoded image can be displayed, for example on media such as posters, billboards, or paper, or on a display of a first device such as smartphone displays, palmtop displays, camera displays, tablet displays, or e-reader displays. A second device can acquire the displayed encoded image, for example by photographing the image. The second device can decode the color space encoded image or transfer the color space encoded image to a device that decodes the image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to the following applications:

U.S. patent application Ser. No. 13/836,447, filed concurrently herewithand entitled “Data Storage and Exchange Device For Color Space EncodedImages;”

U.S. patent application Ser. No. 13/842,932, filed concurrently herewithand entitled “Broadcasting Independent of Network Availability UsingColor Space Encoded Image;”

U.S. patent application Ser. No. 13/837,155, filed concurrently herewithand entitled “Image Encoding and Decoding Using Color Space;”

U.S. patent application Ser. No. 13/837,895, filed concurrently herewithand entitled “Color Restoration for Color Space Encoded Image;”

U.S. patent application Ser. No. 13/842,856, filed concurrently herewithand entitled “Two-Level Error Correcting Codes for Color Space EncodedImage;”

U.S. patent application Ser. No. 13/843,111, filed concurrently herewithand entitled “Frame of Color Space Encoded Image for DistortionCorrection;”

U.S. patent application Ser. No. 13/842,817, filed concurrently herewithand entitled “Information Exchange Using Photo Camera as Display forColor Space Encoded Image;”

U.S. patent application Ser. No. 13/841,338, filed concurrently herewithand entitled “Information Exchange Using Color Space Encoded Image;”

U.S. patent application Ser. No. 13/844,184, filed concurrently herewithand entitled “Large Documents Using Color Space Encoded Image;”

U.S. patent application Ser. No. 13/844,207, filed concurrently herewithand entitled “Combination Book With E-Book Using Color Space EncodedImage;”

U.S. patent application Ser. No. 13/837,769, filed concurrently herewithand entitled “Image Fragmentation for Distortion Correction of ColorSpace Encoded Image;”

U.S. patent application Ser. No. 13/844,000, filed concurrently herewithand entitled “Book Using Color Space Encoded Image;”

U.S. patent application Ser. No. 13/844,127, filed concurrently herewithand entitled “Data Backup Using Color Space Encoded Image;”

U.S. patent application Ser. No. 13/844,168, filed concurrently herewithand entitled “Self-Publication Using Color Space Encoded Image;” and

U.S. patent application Ser. No. 13/840,504, filed concurrently herewithand entitled “Information Broadcast Using Color Space Encoded Image.”

FIELD OF THE DISCLOSURE

The present disclosure relates to applications of and exchange devicesfor color space encoded images, and methods of encoding and decodingcolor space encoded images. More particularly, the present disclosurerelates to encoding and decoding of machine-based data using highcapacity multi-colored composite two-dimensional pictures havingdifferent symbols organized in specific order and sets in a color space.

BACKGROUND

With smartphones and tablets having become widely deployed as well as 3Gand Wi-Fi based access to Internet, data capture technologies work as aninterface between databases and user devices. Example data capturetechnologies include bar codes, QR codes, Radio Frequency Identification(RFID) and Near Field Communication (NFC). Most of these technologiesare used as either a visible or an invisible tag to connect thesedatabases and user devices. The introduction of cloud storage addsextended use of data exchange between data storage and end user devices.

Many households include many different devices simultaneously connectedto Wi-Fi access points. This ubiquitous Wi-Fi usage increaseselectromagnetic fields around the home environment. Despite theusefulness of Wi-Fi, long term consequences for the human body andgeneral health are not clear.

End user devices such as smartphones and tablets are frequently equippedwith photo and/or video cameras. These cameras can be used for capturinginformation presented in different visual forms. The other data capturetechnologies descried earlier, including bar codes and especially twodimensional (2D) bar codes such as QR codes, have attracted theattention of many researchers and companies due to the potential forinexpensive operation.

Bar codes have been used for mobile applications to deliver a multitudeof different mobile services over mobile phones and other mobilecommunication or computing devices. Such applications range fromproviding Uniform Resource Locator (URL) information to link a mobiledevice to the Internet, through to using a bar code as a form ofe-ticket for airlines or event admissions. Hence, there is an evergrowing demand for higher capacity bar codes, suited for robustapplication on mobile devices. Traditional approaches to higher capacitybar codes include: (i) using a colored symbol set and (ii) increasingthe size of the bar code. There are known limitations for eitherapproach, especially related to mobile devices with compromised, lowresolution cameras. The traditional data capacity of 1D and 2D bar codesis severely limited. This severely limited data capacity constrainspossible applications of 1D and 2D bar codes, and their primary task issimply linking camera phones to designated Web sites. Additional tasksmay then be performed based on the Web site. This operation again isbased on using of Wi-Fi or 3G connectivity.

The maximum data capacity of 2D bar codes has been improving over therecent years, resulting in the introduction of newer applications.

Presently, the use of color-based bar code and other symbolic encodingin color space technologies using mobile devices such as camera mobilephones has been limited by the physical size of the actual bar codesymbol in relation to the information encoded within, and also by theimage capturing mechanism on mobile devices to discriminate or resolveeffectively a greater multitude of colors, especially under variedlighting conditions by an inexperienced, lay user. Another limitingfactor has been color fidelity of hard copy output devices, such ascolor printers, in reproducing the true colors of such color based barcode or symbol sets in a color space.

While bar codes can be used to provide 2D encoded data, they have notbeen used to provide real storage devices and media.

SUMMARY

Embodiments of the present invention include systems, methods, andnon-transitory computer program products for information exchange usingcolor space encoded images. A color space encoded image can bedisplayed, for example on media such as posters, billboards, or paper,or on a display of a first device such as smartphone displays, palmtopdisplays, camera displays, tablet displays, or e-reader displays. Asecond device can acquire the displayed encoded image, for example byphotographing the image. The second device can decode the color spaceencoded image or transfer the color space encoded image to a device thatdecodes the image.

In accordance with the disclosed subject matter, methods, devices, andnon-transitory computer program products are provided for exchanginginformation between a first device and a second device.

Certain embodiments include a computer-implemented method for exchanginginformation between a first device and a second device. The methodincludes receiving, at the first device, information for exchanging withthe second device and creating a first image encoding the information bycreating colored cells within a color space to provide an encodedrepresentation of the received information in an image by processing thereceived information to obtain a stream of integers, each integer in thestream of integers corresponding to a color in the color space for acell in the image and creating one or more additional cells adjoiningthe encoded image, the one or more additional cells for assisting indecoding the encoded image. The method also includes displaying thefirst encoded image for a second device to decode and in response to thedisplaying the first encoded image, receiving a second encoded imagefrom the second device, the second encoded image having colored cellscorresponding to a stream of integers encoding a response.

Certain embodiments include a first device for exchanging informationwith a second device. The first device includes memory, storage, and atleast one processor configured to receive, at the first device,information for exchanging with the second device. The at least oneprocessor is further configured to create a first image encoding theinformation by creating colored cells within a color space to provide anencoded representation of the received information in an image byprocessing the received information to obtain a stream of integers, eachinteger in the stream of integers corresponding to a color in the colorspace for a cell in the image and creating one or more additional cellsadjoining the encoded image, the one or more additional cells forassisting in decoding the encoded image. The at least one processor isalso configured to display the first encoded image for a second deviceto decode and in response to the displaying the first encoded image,receive a second encoded image from the second device, the secondencoded image having colored cells corresponding to a stream of integersencoding a response.

The embodiments described herein can include additional aspects of thepresent invention. For example, the method can further include locatingthe second encoded image by locating one or more additional cellsadjoining the second encoded image and decoding the response based onthe colored cells and the one or more additional cells adjoining thesecond encoded image. The at least one processor is further configuredto locate the second encoded image by locating one or more additionalcells adjoining the second encoded image and decode the response basedon the colored cells and the one or more additional cells adjoining thesecond encoded image.

Certain embodiments include a computer-implemented method for exchanginginformation between a first device and a second device. The methodincludes receiving, at the second device, a first encoded image from thefirst device, the first encoded image having colored cells correspondingto a stream of integers encoding data and decoding the first encodedimage by locating the first encoded image by locating one or moreadditional cells adjoining the first encoded image and decoding arequest from the first device based on the colored cells and the one ormore additional cells adjoining the first encoded image. The method alsoincludes determining a response to the first encoded image and creatinga second encoded image encoding the response by creating colored cellswithin a color space to provide an encoded representation of theresponse in the second encoded image by processing the response toobtain a stream of integers, each integer in the stream of integerscorresponding to a color in the color space for a cell in the image andcreating one or more additional cells adjoining the second encodedimage, the one or more additional cells for assisting in decoding thesecond encoded image. The method further includes displaying the secondencoded image for the first device to decode.

Certain embodiments include a first device for exchanging informationwith a second device. The first device includes memory, storage, and atleast one processor configured to receive, at the first device, a firstencoded image from the second device. The first encoded image hascolored cells corresponding to a stream of integers encoding data. Theat least one processor is also configured to decode the first encodedimage by locating the first encoded image by locating one or moreadditional cells adjoining the first encoded image and decoding arequest from the second device based on the colored cells and the one ormore additional cells adjoining the first encoded image. The at leastone processor is further configured to determine a response to the firstencoded image and create a second encoded image encoding the response bycreating colored cells within a color space to provide an encodedrepresentation of the response in the second encoded image by processingthe response to obtain a stream of integers, each integer in the streamof integers corresponding to a color in the color space for a cell inthe image and creating one or more additional cells adjoining the secondencoded image, the one or more additional cells for assisting indecoding the second encoded image. The at least one processor is alsoconfigured to display the second encoded image for the second device todecode.

The embodiments described herein can include additional aspects of thepresent invention. For example, the method can also include receiving,at the first device, the second encoded image to decode, the secondencoded image having colored cells corresponding to a stream of integersencoding the response, locating the second encoded image by locating oneor more additional cells adjoining the second encoded image, anddecoding data based on the colored cells and the one or more additionalcells adjoining the second encoded image. The first device can befurther configured to receive, at the first device, the second encodedimage to decode, the second encoded image having colored cellscorresponding to a stream of integers encoding the response, locate thesecond encoded image by locating one or more additional cells adjoiningthe second encoded image, and decode data based on the colored cells andthe one or more additional cells adjoining the second encoded image.

BRIEF DESCRIPTION OF THE DRAWINGS

Various objects, features, and advantages of the present disclosure canbe more fully appreciated with reference to the following detaileddescription when considered in connection with the following drawings,in which like reference numerals identify like elements. The followingdrawings are for the purpose of illustration only and are not intendedto be limiting of the invention, the scope of which is set forth in theclaims that follow.

FIG. 1 illustrates a system for storing, encoding, and decoding a 2Dcolor space image in accordance with certain embodiments of the presentdisclosure.

FIGS. 2A-2B illustrate the 2D color space encoded image in accordancewith certain embodiments of the present disclosure.

FIG. 3 illustrates a method that the system performs for encoding animage in accordance with certain embodiments of the present disclosure.

FIG. 4 illustrates a method that the system performs for decoding animage in accordance with certain embodiments of the present disclosure.

FIG. 5 illustrates a block diagram of color calibration cells forrestoration of proper color based on distinguishable color samples inaccordance with certain embodiments of the present disclosure.

FIG. 6 illustrates a block diagram of a frame arrangement forgeometrical distortion correction of an encoded image in accordance withcertain embodiments of the present disclosure.

FIG. 7 illustrates a block diagram for frame fragmentation fordistortion correction in accordance with certain embodiments of thepresent disclosure.

FIG. 8 illustrates a block diagram of image based information exchangefrom one device to another device in accordance with certain aspects ofthe present disclosure.

FIG. 9 illustrates a block diagram of an example of information exchangeof a color space encoded image in accordance with certain embodiments ofthe present disclosure.

FIG. 10 illustrates information exchange using a photo camera to displaya color space encoded image in accordance with certain embodiments ofthe present invention.

DETAILED DESCRIPTION

The present systems and methods allow for use of 2D color-space codes toencode images with greater data capacity. The 2D color-space encodedimages described herein can provide a wide variety of applications,regardless of network connectivity. The present systems and methodsallow a sufficient amount of data or even an entire database to beencoded and placed in a storage device using the 2D images encoded andorganized by color-space codes as described herein.

The present disclosure has utility in, for example, data back-up storageapplications or data exchange using mobile devices to provide colortwo-dimensional pictures for processing.

FIG. 1 illustrates a system 100 for storing, encoding, and decoding a 2Dcolor space image in accordance with certain embodiments of the presentdisclosure. System 100 includes a 2D color space image 102. System 100can encode any type of desired machine-based data. Non-limiting examplesof machine-based data include text 104 a, books 104 b (containing, forexample, text and images), databases 104 c, photographs 104 d or otherartwork, images, video, film, or movies, music 104 e, or any other typeof binary data 104 f.

In some embodiments, capturing of desired data can be done using aconventional camera or scanner, or a smartphone or tablet camera.

For example, a user can encode selected information into a 2Dcolor-space coded image and print the image on any desired media (e.g.,paper, textile, plastic, etc.). The 2D color-space coded image can alsobe depicted on any display, including posters, TV screens or computermonitors, as well as displayed on mobile devices such as smartphones ortablets. Digital photo cameras can be used as display devices as well.

Using a capturing device (e.g., a conventional camera or scanner or asmartphone or tablet camera) with an external PC or internal processorhaving image reading or decoding software, the image code can be decodedand the encoded data can be read. The present systems and methods allowinformation to be retrieved, decoded, and saved in the memory of adevice or presented on a display for further use. For example, thedecoded data can be read on a laptop computer 106 a or a desktopcomputer 106 b, a smart phone 106 c, a photo camera 106 d, a palmtop orhandheld device 106 e, or a tablet 106 f. All this can be done withoutmanual work. No network connectivity or additional cost is required.

High Volume Data Storage and Exchange Device and Encoding/DecodingMethods

The present systems and methods relate to a data storage device forstoring 2D color space encoded images, methods of encoding and decodingthe 2D color space encoded images, and application of the 2D color spaceencoded images to data exchange.

More particularly, the present disclosure relates to encoding,transmission, and decoding of machine-based data using an appropriatelyformatted high capacity color image. The present systems and methods canbe utilized, for example, for compact backup of printed materials,and/or data exchange between a color-image-displaying device and aplurality of image-reading devices.

FIGS. 2A-2B illustrate the 2D color space encoded image 102 inaccordance with certain embodiments of the present disclosure. Asillustrated in FIG. 2A, encoded image 102 includes a number ofrectangular cells 202 of different colors. In some embodiments, encodedimage 102 can include a black frame 204, a white frame 206, a “zebraframe” 208 having alternating colors, and color calibration cells 210.

FIG. 2B illustrates a close-up view of the black frame 204, the whiteframe 206, the zebra frame 208, and the color calibration cells 210 inaccordance with certain embodiments of the present disclosure. Blackframe 204 and white frame 206 can be used for rough localization of thedata image including rectangular cells 202, determination of geometricdistortion of the data image, and estimation of cell size and scale ofrectangular cells 202. Zebra frame 208 can be used for furthercalibration of cell size for rectangular cells 202. Color calibrationcells 210 can be used for determining a number of colors N_(c) used forencoding the data, and for indexing the colors to map the colors tonumbers for encoding and decoding.

In some embodiments, rectangular cells 202 can use more than eightdifferent colors. The number of colors can be expressed as N_(c), forexample N_(c)≧8. Preferably, the number of colors is selected to be apower of two.

FIG. 3 illustrates a method 300 that the system performs for encoding animage in accordance with certain embodiments of the present disclosure.

The choice of appropriate color-difference metrics can help achievestable color resolution for image-source media and image-readingdevices. Most imaging devices operating in the visible spectrum areadapted to human eye color resolution. A color space describes colorcapabilities of a particular device by relating numbers to actualcolors. Embodiments of the present disclosure utilize metrics based onlow-cost non-Euclidean distance in red green blue (RGB) color space.

In some embodiments, the present system uses color space metrics whichare close to Commission International de l'eclairage (CIE) 1976(L*,u*,v*) color space metrics. The CIELUV color space distributescolors roughly proportional to perceived color differences. In otherembodiments, the present systems and methods are able to use any colorspace, including the CIE 1931 XYZ color space, CIELAB color space, andCIEUVW color space. Although embodiments of the present system are basedon the CIELUV color space, according to experimentation, the color spaceused in certain embodiments is advantageously less sensitive to a colortemperature of relevant image displaying media or device, andadvantageously less sensitive to external illumination. Assuming R, G,and B are one byte (i.e., 0 . . . 255 or 0x0 . . . 0xFF) normalizedintensity of red, green and blue color components correspondingly, acolor difference ΔC between colors C₁=(R₁, G₁, B₁) and C₂=(R₂, G₂, B₂)can be given by the following equations:

$\begin{matrix}{\mspace{79mu}{{\overset{\_}{r} = \frac{R_{1} + R_{2}}{2}}\mspace{79mu}{{\Delta\; R} = {R_{2} - R_{1}}}\mspace{79mu}{{\Delta\; G} = {G_{2} - G_{1}}}\mspace{79mu}{{\Delta\; B} = {B_{2} - B_{1}}}{{\Delta\; C} = \sqrt{{( {2 + \frac{\overset{\_}{r}}{256}} )\Delta\; R^{2}} + {4\;\Delta\; G^{2}} + {( {2 + \frac{255 - \overset{\_}{r}}{256}} )\Delta\; B^{2}}}}}} & {{Equation}\mspace{14mu}(1)}\end{matrix}$where r describes an average red component value of C₁ and C₂.

The present system first receives machine data for encoding into animage. For example, the machine data can be contained in a file. Thepresent system compresses the file (step 302). For example, the machinedata can be compressed into a binary compressed file such as zip, rar,arc or any other archive format file.

The present system then determines an appropriate color encoding,illustrated by box 310. For example, the present system maps theresulting code sequence to a stream of integers from 0 to N_(c)−1,inclusive. Using the stream of integers, the present system creates abitmap with frame formatting. Generation of frame formatting isdescribed in further detail later. The present system fills the interiorof the frames with color bitmap cells, encoded as described following.In some embodiments, the encoding can use error correction. Anon-limiting example of error correction is the two-stage errorcorrection described herein. In some embodiments, the encoding can usescrambling. In some embodiments, the encoding can use error correctionin combination with scrambling. The present system determines the colorof each cell based on the color calibration cells. The resulting encodedimage can then be sent to any display, including a color printer,display, or other color image output device.

In some embodiments, the present system then performs a first level oferror correction (step 304). One benefit of error correction is that thepresent system is able to decode an image properly, even if a portion ofthe image is damaged. For example, the present system can determine aheader containing a cyclic redundancy check (CRC) error-detecting code,file name, format information, and sequence number (e.g., if the machinedata and resulting encoded image represent only part of a larger datastream such as a video stream). In some embodiments, the present systemcan use any error detecting code such as repetition codes, addition ofparity bits, checksums, hash functions, or more powerful errorcorrecting codes.

In some embodiments, the present system can use a Reed-Solomon encoderof high order. A Reed-Solomon error correction code is a cyclicerror-correcting code capable of detecting and correcting multipleerrors in a bit stream. For example, the present system can use 8-bitwords and blocks of size 255 bytes, if N_(c)=16. In further embodiments,the present system can use other error-correcting codes such aslow-density parity-check (LDPC) codes or turbo codes. Reed-Solomon errorcorrection codes are capable of detecting errors, erasures, andcombinations thereof. In some embodiments, a word size for this highorder level of Reed-Solomon encoding can be equal to k*log₂(N_(c))rounded down, where k>1. As described herein, in some embodiments thetwo-stage error coding can perform high order Reed-Solomon encoding in afirst stage, and low order Reed-Solomon encoding in a second stage. Thesecond stage can also include smaller words. For example, the word sizecan be equal to log₂(N_(c)) bits, rounded down.

In some embodiments, the present system then scrambles the obtained code(step 306). For example, the resulting code can be interleaved using apredefined pseudorandom sequence. The present system scrambles theobtained code to spread out potential errors. Accordingly, the two-levelerror correction described herein can perform better. That is, whenregistering or capturing an image for decoding, there can be colordistorgions due to non-uniform lighting, or damage on the media on whichthe image is printed. For example, the media can have spots, bewrinkled, and the like. The scrambling provides that “bad” color cellswith uncertain data can be distributed uniformly (i.e., independently)into different encoding blocks, thereby decreasing the number of errorsper block. Because error correction can have upper limits of correctableerrors, the present spreading improves the effectiveness of the errorcorrection.

In some embodiments, the present system then performs a second level oferror correction (step 308). For example, the resulting scrambled codeis encoded using a Reed-Solomon encoder with. For the second level oferror correction, the word size can be selected to be smaller than theword size used in the first level of error correction. As describedearlier, this two-stage error correction allows the present system tomark unrecoverable blocks of data during a first stage as erasures. Theerror correction described herein finds an index location of an errorword inside a block, and also corrects the error word to its correct ortrue value. The two-stage error correction allows the present system tocorrect twice as many erasures as errors, as described in more detaillater.

Registration

The present system is able to use any kind of high definition (HD) colorregistration device to register an encoded image. Non-limiting exampleregistration devices can include a digital camera, a smartphone camera,a tablet camera, a video camera, a webcam, a scanner (to scan a hardcopy image), etc.

Decoding

FIG. 4 illustrates a method 400 that the system performs for decoding animage in accordance with certain embodiments of the present disclosure.

Once an encoded image is registered, the present system can copy aregistered image to any processing device containing a decoding module.An example processing device can include a computer, containing adecoding module with computer-readable instructions for decoding, wherethe computer-readable instructions are stored on a non-transitorycomputer storage medium, or containing hardware having hard-coded orembedded decoding instructions stored thereon.

Generally, decoding method 400 includes two high-level processes: (1)image recognition, and (2) two-stage Reed-Solomon decoding. In someembodiments, the present system uses Viterbi decoding as a Reed-Solomondecoder.

The present system locates the main frame (step 402). For example, thepresent system can locate the black frame and the white frame vertices.The present system can further determine the middles of thecorresponding edges of the black frame and the white frame. If the mainframe is not found (step 404: No), the present system returns an error(step 406). For example, the error can be that the image cannot bedecoded, or the error can be more specific such as that the main framecannot be found.

If the main frame is found (step 404: Yes), the present system correctsgeometric distortion based on the main frame and the zebra frame (step412). Geometric distortion can occur when a user uses a camera lens, forexample on a smartphone or tablet camera, or on a digital camera, todecode an image. Accounting for distortion is described in furtherdetail later, in connection with FIG. 6.

The present system locates the color calibration cells (step 408). Asdescribed earlier in connection with FIGS. 2A-2B, in some embodiments,the color calibration cells are located inside the black frame and thewhite frame, on the left of the rectangular cells encoding the machinedata. In other embodiments, the color frame could be located outside theblack frame, between the black and white frames, or in other locationswithin the white frame.

Based on the color frame, the present system decodes the colors in therectangular cells (step 410). In general, for every pixel found in theencoded image, the present system finds the closest sample pixel basedon the color calibration cells and substitutes the sample pixel for thedetected pixel. In some embodiments, the present system obtains acolor-value correspondence. The present system determines cell sizecalibration based on the zebra frame. The present system divides thedata image into color cells. When decoding the image, the present systemcaptures or collects all colors belonging to a given pixel rectangle. Ifmost of the pixel colors in the pixel rectangle could be recognized asone color from the color calibration cells, that color from the colorcalibration cells is used. Otherwise, the present system marks the pixelas an erasure. If a pixel-level block cannot be decoded, the presentsystem treats the whole pixel block as an erasure. The present systemreads the rectangular cells sequentially and determines an index of thenearest color from the color calibration cells to the median cell color.In particular, the present system determines the nearest color accordingto nearest Euclidean distance in the appropriate color space, accordingto Equation (1), described earlier.

The present system decodes the bytes (step 414). In some embodiments,the present system applies a corresponding decoder to theerror-correcting encoder described earlier. For example, the presentsystem applies a low-level Reed-Solomon decoder to the code streamobtained from the decoded colors. Accordingly, due to the nature of theerror-correcting code, the present system is able to restore erasuresand errors in the code stream. If the present system detectsunrepairable blocks, the unrepairable blocks are marked as “erasures.”

The present system then descrambles the bytes (step 416). In someembodiments, the present system de-interleaves the resulting bytestream.

The present system then decodes the original information based on thedecoded byte stream (step 418). In general, the present system uses acorresponding decoder to the error-correcting encoder described earlier,to restore pixel-block-level erasures and errors. In some embodiments,the present system applies a high-level Reed-Solomon decoder. Afterapplying the high-level Reed-Solomon decoder, if some pixel blocksremain un-repairable, the present system attempts to change the indicesto those indices corresponding to the next nearest color in the colorcalibration cells, and then repeats Reed-Solomon decoding of themodified block. If error free data is obtained, the present system mapsthe resulting sequence of integers to a bit stream. The present systemthen parses the resulting bit stream and computes a cyclic redundancycheck (CRC) of the data. The present system compares the resulting CRCwith the CRC contained in the header of the image. The present systemthen saves or appends the file. Two-level COLOR error correction schemawith erasures and statistical color restoration

The two-level error correction used in the present system and describedearlier in connection with FIGS. 3 and 4 allows the present system tocorrect two types of errors: errors and erasures. In some embodiments,the Reed-Solomon error-correction encoding used is able to detect errors(which correspond to wrong, incorrect, or unexpected information) anderasures (which correspond to absent or missing information).

In some embodiments, correcting erasures can be performed twice aseffectively than correcting errors. To make use of this feature, thepresent system uses a two-level error correction schema.

For example, assume N_(c)=16 (i.e., 2⁴). Accordingly, there are sixteendistinctive colors which allow an encoded image to encode 4 bits ofinformation per rectangular cell. Every byte of information (8 bits) isencoded by two color indices. The error correction scheme has twolevels. First, the byte array is divided into chunks, and every chunk isappended with Reed-Solomon source control data. Second, every byte isdivided into two half-bytes (4 bits). The half-bytes are then gatheredinto groups or chunks which are appended with Reed-Solomon sourcecontrol data. For example, the present system can process <N_(c)−1half-bytes. That is, if N_(c)=16, the present system gathers less thanfifteen half-bytes. The group of half-bytes is then appended withReed-Solomon source control data determined based on the group ofhalf-bytes. Because the Reed-Solomon source control data can be aboutthe same size as the input data, the addition of the Reed-Solomon sourcecontrol data results in the size of the group reaching fifteen bytes.For example, the size of the input data and the control data block sizecan be fifteen bytes. Each half-byte in the group is mapped to anappropriate color, for example by using the corresponding half-bytes asindices into the color array. Accordingly, the encoding process appliestwo levels of Reed-Solomon encoding, first at the byte array level andsecond at the half-byte level.

As described earlier, when decoding the image, the present systemcollects all colors belonging to a detected pixel rectangle. If most ofthe colors in the pixel rectangle could be recognized as one of thecolors in the color calibration cells, the present system uses thecorresponding color in the color calibration cells. Otherwise, the pixelrectangle is marked as an erasure. If a pixel-level block cannot bedecoded, the whole pixel block is treated as an erasure. As describedearlier, the erasures can be further restored by an appropriateerror-correction algorithm. The two-level error correction andleveraging of erasure determinations significantly improves the qualityof the decoding described earlier in connection with FIG. 4. Inparticular, the two-level error correction described herein allows thepresent system to recover information which otherwise would be lost.

Restoration of Proper Color in the Case of Distinguishable Color Samples

FIG. 5 illustrates a block diagram of color calibration cells forrestoration of proper color based on distinguishable color samples inaccordance with certain embodiments of the present disclosure. FIG. 5includes color calibration cells 210.

Color calibration cells 210 include all colors used to encode therectangular cells. In some embodiments, a white frame or bordersurrounds color calibration cells 210 for the color white. In otherembodiments, the present system may also include white as a color blockin the color calibration cells arranged on the left hand side. Thepresent system discerns each color based on its placement within colorcalibration cells 210. The present system then stores distorted pixelsas color samples. Different variants of a distorted pixel are saved topoint or refer to the same correct pixel. That is, distorted pixels aredetermined dynamically, based on the position of their colors among thecolor calibration cells. The distorted pixels are assigned estimatedcolors according to the color calibration cells.

As described earlier, during decoding (shown in FIG. 4), the presentsystem translates a distorted pixel into one of the original colors incolor calibration cells 210 using the nearest original color to thedistorted pixel. For example, let color C=(R, G, B) represent a redgreen blue (RGB) component representation of the color. According toEquation (1), described earlier, a pseudo-Euclidean distance between twocolors C₁ and C₂ can be given by ΔC, as described in Equation (1):

${\Delta\; C} = \sqrt{{( {2 + \frac{\overset{\_}{r}}{256}} )\Delta\; R^{2}} + {4\;\Delta\; G^{2}} + {( {2 + \frac{255 - \overset{\_}{r}}{256}} )\Delta\; B^{2}}}$

This distance generates a native pseudo-Euclidean metric in the colorspace. A color mapping can be represented by the set of pairs:(distorted color)→(correct color). The distorted color can be C₁, andthe correct color can be C₂. The number of such color pairs should begreater than or equal to N_(c), the total number of colors. This mappingcan be determined based on the color calibration cells and on the colorarrangement therein.

To restore a correct or proper color c* from a distorted color, thepresent system iterates over all pairs of distorted colors and correctcolors. In particular, the present system selects the pair in which thedistorted color portion of the pair is closest to the correct or propercolor c*. In some embodiments, the determination of whether thedistorted color portion is closest to the correct or proper color ismade according to the pseudo-Euclidian metrics in the RGB color space.That is, the present system selects the pair in which ΔC is smallestaccording to Equation (1). The selected pair provides the correct color.Accordingly, the present system allows the correct colors to berestored, because the present system saves the correspondence betweenthe distorted values and the original values. Therefore, the presentsystem is able to leverage information about the way in which the colorswere distorted relative to the original values as captured in the colorcalibration cells.

Frame Arrangement for Automatic Image Geometrical Distortion Correction

FIG. 6 illustrates a block diagram of a frame arrangement forgeometrical distortion correction of an encoded image in accordance withcertain embodiments of the present disclosure.

In some embodiments, the present system includes arrangements torecognize placement of rectangular cells and other elements in theencoded image. For example, these arrangements can include black frame204, white frame 206, zebra frame 208, and color calibration sidebar 210(shown in FIG. 2). The black frame and white frame surround the encodedrectangular cells. The color calibration sidebar contains sample colors(described earlier, in connection with FIG. 5). The zebra framesurrounds the encoded rectangular cells to allow the present system todetermine appropriate resolution and pixel division of the encodedimage.

The black frame and white frame allow the present system to determineand correct nonlinear distortion. As described earlier, nonlineardistortion can be introduced, for example, by camera lenses whenregistering an encoded image for decoding. In some embodiments, thepresent system uses the white frame as follows. The white frame can berestricted by curve x_(left)(y) 602, curve x_(right)(y) 604, curvey_(top)(x) 606, and curve y_(bottom)(x) 608. For every point (x*,y*) 610which belongs to the area restricted by curves 602-608, the presentsystem can determine parameters a and b (0≦a≦1 and 0≦b≦1) according tothe following equations:x _(left)(y*)+ax _(right)(y*)=*x  (1-a)y _(top)(x*)+by _(bottom)(x*)=y*  (1-b)The equation described above maps the area restricted by curves 602-608to a unit rectangle defined by 0≦a≦1, 0≦b≦1. The unit rectangle canrepresent what the image would look like when “straightened out,” bymapping the area restricted by curves 602-608 to the unit rectangle.That is, the distorted boundary of the image determined according to thearea restricted by curves 602-608 can be mapped to the “straightenedout” unit rectangle. The zebra frame can be used to determine parametersa and b. The present distortion correction therefore allows the originalarea having curved or non-linear sub-regions to be transformed accordingto the unit rectangle using parameters a and b.

The zebra frame allows the present system to determine a resolution ofthe embedded picture, and to divide the image into “bold” or unknownpixels. As described earlier, the color value of some pixels may beunknown. “Bold” pixels allow the present system to determine the propercolor value of a pixel according to a voting process. In someembodiments, the voting process can choose the color value of the mostfrequent pixel close to the “bold” or unknown pixel to be the colorvalue of the “bold” pixel. Otherwise, if the result of the voting isunclear—for example, if there is no pixel with a frequency much largerthan the other pixels—the present system can mark the pixel as anerasure (i.e., missing). Therefore, using the two-level error correctiondescribed earlier, the pixel can be restored.

Image Fragmentation

FIG. 7 illustrates a block diagram for frame fragmentation fordistortion correction in accordance with certain embodiments of thepresent disclosure. The rectangular cells can be divided into sub-images704 using separating lines 702.

Frame fragmentation or image fragmentation refers to a complementarytechnique to distortion correction (shown in FIG. 6). In someembodiments, dividing the image into a set of sub-images 704 allows thepresent system to work with every sub-image 704, without requiring thedistortion compensation technique described earlier because sub-images704 are sufficiently small that there is not as much distortion as withthe entire encoded image. In some embodiments, sub-images 704 can beseparated with separating lines 702. In some embodiments, separatinglines 702 can be black lines, which can also became part of the zebraborder.

The present system can use the same encoding and decoding techniquesdescribed earlier (shown in FIGS. 3 and 4) for encoding and decoding ofsub-images 704. For example, the present system can leverage thetwo-level error correction, scrambler, etc. described earlier.

In some embodiments, the present system can skip the distortioncorrection described earlier (shown in FIG. 6). The present system canleverage the relatively smaller size of sub-images 704 to skipnon-linear distortion compensation of each sub-image 704. Accordingly,the present system can decode sub-images 704 with better accuracy.

“Image” Based Data Exchange Between Devices without Using Wi-Fi orCellular

FIG. 8 illustrates a block diagram of image based information exchangefrom one device to another device in accordance with certain aspects ofthe present disclosure. FIG. 8 includes smartphones 802 a-b, and encodedimages 804 a-b. Some smartphones currently have large displays and manyare evolving to include even larger displays. These displays can be usedfor information exchange, for example using color-space encoded images804 a-b. The color-space encoded images 804 a-b can be encoded accordingto the encoding process described earlier, in FIG. 3. Accordingly,smartphones 802 a-b are able to exchange information using the encodedimages. For example, an embedded camera on smartphone 802 b can registeror capture an encoded image 804 a being displayed on smartphone 802 a.

Embedded cameras used on devices such as smartphones 802 a-b areimproving. As described herein, the resolution per frame as captured byembedded cameras can be less important, because the requirements on thenumber of pixels which can be pixel number limitation can be avoided.Descriptions of the pixel number limitation are described in furtherdetail below.

In some embodiments, smartphones 802 a-b can have hardware modules orsoftware applications loaded thereon, for example for encoding ordecoding according to the processes described in FIGS. 3 and 4. A firstuser can create machine data or otherwise store machine data on a devicesuch as a smartphone. The first user can then decide to exchange relatedinformation between devices, for example with a second user. The machinedata created by the first user or stored on the phone can be encodedaccording to the process described in FIG. 3, and then displayed on thesmartphone display for exchanging information with smartphone 802 b.

As described earlier, smartphone 802 b can acquire the encoded imagethrough its embedded camera. Advantageously, this acquisition does notrequire smartphone 802 b to download or acquire encoded image 804 athrough electromagnetic radio transmissions such as a wireless orcellular network. Smartphone 802 b can perform the decoding as describedin FIG. 4 to determine the machine data from the first user. Asdescribed earlier, the decoding can be performed in hardware or insoftware installed on smartphone 802 b. The machine data encoded by thefirst user can be displayed on the screen or otherwise stored insmartphone 802 b.

In some embodiments, security can be added to the encoded image. Forexample, when the present system compresses or archives the machinedata, the present system can add a password or otherwise encrypt therelevant machine data.

In some embodiments, if there are low light conditions and/or low cameraresolution for smartphone 802 b to register or otherwise capture encodedimage 804 a, smartphone 802 a can partition or divide the information tobe transmitted. For example, smartphone 802 a can transmit a first frameincluding only partial information. In some instances, the first framecan include the color calibration frame for smartphone 802 b tocalibrate colors in preparation for decoding encoded image 804 a, asdescribed earlier in connection with FIG. 5. In some embodiments, thefirst frame can include the main frames (i.e., the black frame and/orthe white frame) to correct geometrical distortions, etc., as describedearlier in connection with FIG. 6. Generally, smartphone 802 a canconstruct and generate special frames for partial transmission. Thespecial frames can include the color calibration frame, the main frames,or the zebra frame, to obtain a stable information and data exchange.

In some embodiments, smartphones 802 a-b can be placed substantiallyagainst each other, for example by aligning the embedded camera ofsmartphone 802 b with the display of smartphone 802 a displaying encodedimage 804 a. Accordingly, smartphones 802 a-b can sense the proximity toestablish automatically the data transmission channel described earlier.In further embodiments, smartphones 802 a-b can change the position ofwhich smartphone represents the source for sending machine data, andwhich smartphone represents the sink for receiving machine data.

Direct Data Exchange

FIG. 9 illustrates a block diagram of an example of information exchangeof a color space encoded image in accordance with certain embodiments ofthe present disclosure. FIG. 9 includes a number of devices includingsmartphones 906 a, 912 a, camera 906 b, 912 b, palmtop or handhelddevice 906 c, 912 c, and tablet or e-reader 906 d, 912 d. In someembodiments, the present system can include information exchange betweena number of devices which can register, capture, or “see” an encodedimage 102 on a display 902. The devices which can register encoded image102 include devices with embedded cameras 904. For example, the devicescan include smartphone 912 a, camera 912 b, palmtop or handheld device912 c, and tablet or e-reader 912 d. In some embodiments, these devicescan be similar to devices 106 c-f (shown in FIG. 1). These registeringdevices can register or capture the corresponding data source. In someembodiments, the information exchange described herein can include datadistribution including multiple devices. Accordingly, the informationexchange described herein can include point-to-multipoint datadistribution, in which a number of devices all capture a single encodedimage 102.

In some embodiments, the present system can include sequential frametransmission of multiple encoded images. The encoded images can includemultiple video frames, such as video frames 910 a-c. Accordingly, thepresent system allows exchange of video files such as machine dataincluding movies or films 908.

In some embodiments, the sequential frame transmission described hereincan be used for machine data other than video files. For example,multiple encoded images can be used to encode large files and/or largeamounts of information. Smartphones such as smartphones 906 a, 912 a arecapable of recording full high definition (HD) video. HD video caninclude video at 1920×1080 pixels, and at least 25 frames per second(fps). Accordingly, smartphones 912 a can often include high-performancecameras such as for capturing photographs or video. Furthermore,smartphone displays continue to improve, allowing for even moreresolution for encoding machine data, and for displaying andtransmitting encoded images.

Smartphones often include a second camera, for example on the front ofthe smartphone along with the display, for use in videoconferencing. Insome embodiments, the present system can allow two devices to be placedwith their displays next to or adjacent with each other. In such anorientation, embodiments of the present disclosure can use the frontcamera. Advantageously, having devices paired as described herein canallow the devices to exchange information interactively. For example, afirst device, such as tablet 906 d can display a first frame. Asdescribed earlier, the first frame can include color calibrationinformation. When the second device has completed processing the colorcalibration information, the second device can encode an acknowledgementin an encoded image of its own (not shown), and send the acknowledgementfor the first device to decode. In some embodiments, the second devicecan send a request for the next frame or additional partial information.

In further embodiments, the present system can send a number of encodedimages, for example in a video-like transmission. This video-liketransmission can be used for two-way interactive or one-way directionalinformation or data exchange.

FIG. 10 illustrates information exchange using a photo camera to displaya color space encoded image in accordance with certain embodiments ofthe present invention. FIG. 10 illustrates a photo camera 1002 and asmartphone 802 b. Photo camera 1002 can display an encoded image 804 a.Current digital cameras such as photo camera 1002 have good optics intheir camera lenses. Current digital cameras often have good displays aswell, which can be used for displaying encoded image 804 a.

Camera manufacturers are also adding wireless connectivity such as Wi-Fior cellular connectivity, or even GPS receivers to make cameras“connected.” Cameras can also include digital signal processors used forimage data processing.

In some embodiments, the present system includes digital signalprocessors modified in photo camera such as photo camera 1002. Themodifications can include hardware modules or embedded software such asfirmware for encoding or decoding color space images in accordance withthe processes described in FIGS. 3 and 4.

Accordingly, photo cameras can be modified for information or dataexchange between cameras. A camera is a good “image” acquisition,registration, or capture device, due to high quality optics and abilityto work under difficult light conditions. Information acquired by acamera can often be stored on a memory card, for example flash memory.As described earlier, the present system can allow a user to insert amemory card into a compatible device, such as a PC or notebook computer.Modules on the PC or notebook computer can decode a color space encodedimage without requiring radio-related approaches such as using a Wi-Fior cellular network.

In some embodiments, an embedded camera on a PC or tablet can be used asan image acquisition, registration, or capture device. In furtherembodiments, the present system can use the display on the PC or tabletas a data source device as well, for example to display encoded image804. Accordingly, the decoding modules described earlier can be utilizedon corresponding decoding devices.

Although specific steps are disclosed in FIGS. 3 and 4, such steps areexemplary. That is, the present system is well-suited to performingvarious other steps or variations of the steps recited in FIGS. 3 and 4.The steps in FIGS. 3 and 4 may be performed in an order different thanpresented, and that not all of the steps may be performed. FIGS. 3 and 4include processes that, in various embodiments, are carried out by aprocessor under the control of computer-readable and computer-executableinstructions. Embodiments of the present disclosure may thus be storedas non-transitory computer-readable media or computer-executableinstructions including, but not limited to, firmware updates, softwareupdate packages, or hardware (e.g., ROM).

Reference has been made in detail to various embodiments in accordancewith the present disclosure, examples of which are illustrated in theaccompanying drawings. While the invention has been described inconjunction with various embodiments, these various embodiments are notintended to limit the invention. On the contrary, the invention isintended to cover alternatives, modifications, and equivalents, whichmay be included within the scope of the invention as construed accordingto the appended claims. Furthermore, in the detailed description ofvarious embodiments, numerous specific details have been set forth inorder to provide a thorough understanding of the invention. However, theinvention may be practiced without these specific details. In otherinstances, well known methods, procedures, components, and circuits havenot been described in detail, so as not to unnecessarily obscure aspectsof the invention.

Some portions of the detailed descriptions have been presented in termsof procedures, logic blocks, processing, and other symbolicrepresentations of operations on data bits within a non-transitorycomputer memory. These descriptions and representations are the meansused by those skilled in the data processing arts to most effectivelyconvey the substance of their work to others skilled in the art. In thepresent disclosure, a procedure, logic block, process, or the like, isconceived to be a self-consistent sequence of operations or steps orinstructions leading to a desired result. The operations or steps arethose utilizing physical manipulations and transformations of physicalquantities. Usually, although not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared, and otherwise manipulated in a computersystem or computing device. It has been convenient at times, principallyfor reasons of common usage, to refer to these signals as records,transactions, bits, values, elements, symbols, characters, samples,pixels, or the like.

Of course, all of these and similar terms are to be associated with theappropriate physical quantities and are merely convenient labels appliedto these quantities. Unless specifically stated otherwise as apparentfrom the foregoing discussions, it is appreciated that throughout thepresent disclosure, discussions utilizing terms such as “deactivating,”“disabling,” “re-activating,” “enabling,” “sending,” “receiving,”“determining,” “flushing,” “responding,” “generating,” “making,”“blocking,” “accessing,” “taking a snapshot,” “associating,” “allowing,”“updating,” or the like, refer to actions and processes of a computersystem or similar electronic computing device or processor. The computersystem or similar electronic computing device manipulates and transformsdata represented as physical (electronic) quantities within the systemmemories, registers or other such information storage, transmission ordisplay devices.

The present systems and methods can be implemented in a variety ofarchitectures and configurations. For example, the present systems andmethods can be implemented as part of a distributed computingenvironment, a cloud computing environment, a client server environment,etc. The embodiments described herein may be discussed in the generalcontext of computer-executable instructions residing on some form ofnon-transitory computer-readable storage medium, such as programmodules, executed by one or more computers, computing devices, or otherdevices. As described earlier, non-limiting examples ofcomputer-readable storage media may include storage media andcommunication media. Generally, program modules include routines,programs, objects, components, data structures, etc., that performparticular tasks or implement particular abstract data types. Thefunctionality of the program modules may be combined or distributed asdesired in various embodiments.

The foregoing descriptions of specific embodiments of the presentsystems and methods have been presented for purposes of illustration anddescription. The specific embodiments are not intended to be exhaustiveor to limit the invention to the precise forms disclosed, and manymodifications and variations are possible in light of the abovedescription. The embodiments were chosen and described in order to bestexplain the principles of the invention and its practical application,to thereby enable others skilled in the art to best utilize theinvention and various embodiments with various modifications as aresuited to the particular use contemplated. It is intended that the scopeof the invention be defined by the claims appended hereto and theirequivalents.

What is claimed is:
 1. A computer-implemented method for exchanginginformation between a first device and a second device, the methodcomprising: receiving, at the first device, information for exchangingwith the second device; creating a first image encoding the informationby: creating colored cells within a color space to provide an encodedrepresentation of the received information in an image by processing thereceived information to obtain a stream of integers, each integer in thestream of integers corresponding to a color in the color space for acell in the image; and creating one or more additional cells adjoiningthe encoded image, the one or more additional cells for assisting indecoding the encoded image; displaying the first encoded image for asecond device to decode; in response to the displaying the first encodedimage, receiving a second encoded image from the second device, thesecond encoded image having colored cells corresponding to a stream ofintegers encoding a response; locating the second encoded image bylocating one or more additional cells adjoining the second encodedimage; and decoding the response based on the colored cells and the oneor more additional cells adjoining the second encoded image.
 2. Acomputer-implemented method of exchanging information between a firstdevice and a second device, the method comprising: receiving, at thesecond device, a first encoded image from the first device, the firstencoded image having colored cells corresponding to a stream of integersencoding data; decoding the first encoded image by locating the firstencoded image by locating one or more additional cells adjoining thefirst encoded image; and decoding a request from the first device basedon the colored cells and the one or more additional cells adjoining thefirst encoded image; determining a response to the first encoded image;creating a second encoded image encoding the response by: creatingcolored cells within a color space to provide an encoded representationof the response in the second encoded image by processing the responseto obtain a stream of integers, each integer in the stream of integerscorresponding to a color in the color space for a cell in the image; andcreating one or more additional cells adjoining the second encodedimage, the one or more additional cells for assisting in decoding thesecond encoded image; displaying the second encoded image for the firstdevice to decode and the one or more additional cells adjoining thesecond encoded image, so as to permit the first device to locate thesecond encoded image by locating the one or more additional cells and todecode the second encoded image.
 3. A device for exchanging informationwith a second device, the first device comprising: one or more displays;memory; storage; and at least one processor configured to use the memoryand the storage to: receive, at the device, information for exchangingwith the second device; create a first image encoding the informationby: creating colored cells within a color space to provide an encodedrepresentation of the received information in an image by processing thereceived information to obtain a stream of integers, each integer in thestream of integers corresponding to a color in the color space for acell in the image; and creating one or more additional cells adjoiningthe encoded image, the one or more additional cells for assisting indecoding the encoded image; display, on the one or more displays, thefirst encoded image for a second device to decode; in response to thedisplaying the first encoded image, receive a second encoded image fromthe second device, the second encoded image having colored cellscorresponding to a stream of integers encoding a response; locate thesecond encoded image by locating one or more additional cells adjoiningthe second encoded image; and decode the response based on the coloredcells and the one or more additional cells adjoining the second encodedimage.
 4. A device for exchanging information with a second device, thefirst device comprising: one or more displays; memory; storage; and atleast one processor configured to use the memory and the storage to:receive, at the first device, a first encoded image from the seconddevice, the first encoded image having colored cells corresponding to astream of integers encoding data; decode the first encoded image bylocating the first encoded image by locating one or more additionalcells adjoining the first encoded image; and decoding a request from thesecond device based on the colored cells and the one or more additionalcells adjoining the first encoded image; determine a response to thefirst encoded image; create a second encoded image encoding the responseby: creating colored cells within a color space to provide an encodedrepresentation of the response in the second encoded image by processingthe response to obtain a stream of integers, each integer in the streamof integers corresponding to a color in the color space for a cell inthe image; and creating one or more additional cells adjoining thesecond encoded image, the one or more additional cells for assisting indecoding the second encoded image; display, on the one or more displays,the second encoded image for the second device to decode and the one ormore additional cells adjoining the second encoded image, so as topermit the second device to locate the second encoded image by locatingthe one or more additional cells and to decode the second encoded image.